RHAIENG-4645: add integration test for CrewAI websearch agent by mpk-droid · Pull Request #100 · red-hat-data-services/agentic-starter-kits

mpk-droid · 2026-05-13T18:15:53Z

Ticket

Summary

Add deployment integration test (test_deployment.py) for the CrewAI websearch agent, consistent with the existing LangGraph agent tests
Add test-integration Makefile target with proper PYTHONPATH for shared utils
Exclude tests/integration/ from the unit test target
Add httpx>=0.27 dev dependency (required by shared integration utils)
Add crewai-websearch-agent to the nightly CI workflow matrix

Test plan

make test from agents/crewai/websearch_agent/ still runs unit tests without integration tests
make test-integration runs the deployment test — requires cluster auth, will be verified by nightly CI
Integration test builds image, deploys via Helm, validates /health returns 200, and tears down via make undeploy — requires cluster auth, will be verified by nightly CI
JUnit XML output (results.xml) is configured via --junitxml=results.xml in the Makefile target

🤖 Generated with Claude Code

Add deployment integration test that builds, deploys, health-checks, and tears down the CrewAI websearch agent on OpenShift — consistent with the existing LangGraph agent integration tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

coderabbitai · 2026-05-13T18:17:07Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: f2cc42be-8cdd-4b4e-a27e-2ea81a97b69e

📥 Commits

Reviewing files that changed from the base of the PR and between 3f0effc and 2adc749.

📒 Files selected for processing (2)

.gitignore
agents/crewai/websearch_agent/Makefile

✅ Files skipped from review due to trivial changes (1)

.gitignore

🚧 Files skipped from review as they are similar to previous changes (1)

agents/crewai/websearch_agent/Makefile

📝 Walkthrough

Walkthrough

Adds the CrewAI websearch agent to the CI integration test matrix and implements an integration test that builds, deploys, verifies the agent's /health endpoint, and performs teardown on an OpenShift cluster.

Changes

CrewAI Websearch Agent Integration Testing

Layer / File(s)	Summary
Workflow Matrix Integration `.github/workflows/agent-deployment-test.yaml`	`crewai-websearch-agent` is added to the `test-agent` job matrix entries so the agent is deployed and validated on OpenShift during CI.
Test infra, Makefile & deps `agents/crewai/websearch_agent/Makefile`, `agents/crewai/websearch_agent/pyproject.toml`, `agents/crewai/websearch_agent/tests/integration/conftest.py`, `.gitignore`	Makefile splits unit vs integration tests and adds `test-integration`; `httpx>=0.27` added to dev optional dependencies; shared pytest fixtures (`cluster_auth`, `repo_root`) are re-exported for local discovery; `results.xml` is ignored.
Deployment test implementation `agents/crewai/websearch_agent/tests/integration/test_deployment.py`	New integration test module adds fixtures for agent path/name, writes a `.env` from required env vars, runs `build-openshift` and `deploy`, retrieves the external route, validates `/health` returns `healthy` and `agent_initialized=True`, and undeploys in teardown.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: adding an integration test for the CrewAI websearch agent, matching the primary objective of the pull request.
Description check	✅ Passed	The description is well-structured with ticket reference, summary of changes, and test plan, all directly related to the changeset. It clearly explains what was added and why.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/RHAIENG-4645-crewai-websearch-integration-test

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

mpk-droid · 2026-05-13T18:39:15Z

Missing from CI workflow matrix — the crewai-websearch-agent isn't in the test-agent job matrix in .github/workflows/agent-deployment-test.yaml, so this integration test will never run in nightly CI. AC #7 ("Test passes in the nightly CI pipeline") isn't satisfied without it.

Suggested addition to the matrix:

- { name: crewai-websearch-agent, dir: agents/crewai/websearch_agent }

Missing httpx dev dependency — pyproject.toml only has pytest in [project.optional-dependencies].dev, but tests/integration/utils.py imports httpx for the health check. make test-integration will fail with ImportError. The LangGraph react_agent includes httpx>=0.27 in its dev deps — this agent should too.

Assisted by Claude Opus 4.6 (1M context)

Address PR review comments: - Add httpx>=0.27 to dev dependencies (required by shared integration utils) - Add crewai-websearch-agent to the nightly CI workflow matrix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

mpk-droid · 2026-05-13T18:42:20Z

Both comments addressed in 3f0effc:

CI workflow matrix — added { name: crewai-websearch-agent, dir: agents/crewai/websearch_agent } to the test-agent matrix in .github/workflows/agent-deployment-test.yaml.
httpx dev dep — added httpx>=0.27 to [project.optional-dependencies].dev in pyproject.toml, matching the LangGraph agents.

Assisted by Claude Opus 4.6 (1M context)

andrewdonheiser · 2026-05-14T12:27:58Z

[Important] Makefile formatting inconsistency with established agents agents/crewai/websearch_agent/Makefile:171-172

The test-integration target puts the entire command on a single line, while both react_agent and HITL use backslash line continuations for readability. The help text also says "tests" (plural) vs "test" (singular) in the other two.

Current:

test-integration: ## Run integration deployment tests
	PYTHONPATH=$$(git rev-parse --show-toplevel)/tests uv run --extra dev python -m pytest tests/integration/test_deployment.py -v --tb=long --junitxml=results.xml

Expected (matching react_agent/HITL):

test-integration: ## Run integration deployment test
	PYTHONPATH=$$(git rev-parse --show-toplevel)/tests \
	  uv run --extra dev python -m pytest tests/integration/test_deployment.py \
	    -v --tb=long --junitxml=results.xml

Suggested fix: Reformat to match the react_agent/HITL style with backslash line continuations and singular "test".

andrewdonheiser · 2026-05-14T12:28:00Z

[Important] results.xml not in .gitignore

The --junitxml=results.xml flag produces a local file. If someone runs make test-integration locally, results.xml could get accidentally committed. The root .gitignore does not have a results.xml or *.xml entry.

Suggested fix: Add results.xml to .gitignore. This affects all agents, not just this PR — could be a follow-up if preferred.

Address review feedback: - Reformat test-integration target with backslash line continuations and singular "test" to match react_agent/HITL style - Add results.xml to root .gitignore to prevent accidental commits Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

mpk-droid · 2026-05-14T18:07:43Z

Both addressed in 2adc749:

Makefile formatting — reformatted test-integration with backslash line continuations and singular "test" to match react_agent/HITL.
results.xml in .gitignore — added to root .gitignore so it covers all agents.

Assisted by Claude Opus 4.6 (1M context)

github-actions Bot added area/crewai size/s labels May 13, 2026

mpk-droid commented May 13, 2026

View reviewed changes

Comment thread agents/crewai/websearch_agent/tests/integration/test_deployment.py

github-actions Bot added area/ci size/m and removed size/s labels May 13, 2026

mpk-droid marked this pull request as ready for review May 13, 2026 18:46

andrewdonheiser self-requested a review May 14, 2026 12:01

andrewdonheiser approved these changes May 14, 2026

View reviewed changes

mpk-droid merged commit a327361 into main May 14, 2026
6 checks passed

mpk-droid deleted the feat/RHAIENG-4645-crewai-websearch-integration-test branch May 14, 2026 19:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RHAIENG-4645: add integration test for CrewAI websearch agent#100

RHAIENG-4645: add integration test for CrewAI websearch agent#100
mpk-droid merged 3 commits into
mainfrom
feat/RHAIENG-4645-crewai-websearch-integration-test

mpk-droid commented May 13, 2026 •

edited by atlassian Bot

Loading

Uh oh!

coderabbitai Bot commented May 13, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

mpk-droid commented May 13, 2026

Uh oh!

Uh oh!

mpk-droid commented May 13, 2026

Uh oh!

andrewdonheiser commented May 14, 2026

Uh oh!

andrewdonheiser commented May 14, 2026

Uh oh!

mpk-droid commented May 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mpk-droid commented May 13, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Ticket

Summary

Test plan

Uh oh!

coderabbitai Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

mpk-droid commented May 13, 2026

Uh oh!

Uh oh!

mpk-droid commented May 13, 2026

Uh oh!

andrewdonheiser commented May 14, 2026

Uh oh!

andrewdonheiser commented May 14, 2026

Uh oh!

mpk-droid commented May 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mpk-droid commented May 13, 2026 •

edited by atlassian Bot

Loading

coderabbitai Bot commented May 13, 2026 •

edited

Loading